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ABSTRACT 

The Leadership to Unify School Improvement Efforts (LUSIE) 
project worked with school leadership teams from 11 schools in 3 West 
Virginia county districts to create school plans for improvement. The school 
leadership teams, a total of 66 people representing a cross-section of each 
school community, accomplished this at four retreats held in Charleston, West 
Virginia, in 1996 and 1997. At the fourth retreat, LUSIE participants wrote a 
guide for other West Virginia schools involved in creating state-required 
Unified School Improvement Plans. Telephone interviews conducted with 23 
participants found that most participants seemed pleased with LUSIE and their 
resultant school plans, although some expressed skepticism concerning 
implementation of the plans. Many noted their appreciation of the opportunity 
to communicate with other educators and people concerned about education. 
Several respondents said they had developed new competence in communicating 
and organizing change. The three students interviewed felt the proceedings 
were irrelevant or ineffective. The evaluation resulted in the following 
recommendations: integrate students more fully into LUSIE activities; 
initiate follow-up activities to support schools in the implementation of 
their plans; and address the concerns of those who noted a dis juncture 
between theoretical considerations and the practical applications of the work 
accomplished. Two appendices present the interview protocol and the 
evaluation standards checklist. (TD) 
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AEL's mission is to link the knowledge from research with the wisdom from practice to improve 
teaching and learning. AEL serves as the Regional Educational Laboratory for Kentucky, Termessee, 
Virginia, and West Virginia. For these same four states, it operates both a Regional Technology 
Consortium and the Eisenhower Regional Consortium for Mathematics and Science Education. In 
addition, it serves as the Region IV Comprehensive Center and operates the ERIC Clearinghouse on 
Rural Education and Small Schools. 

Information about AEL projects, programs, and services is available by writing or calling AEL. 





Appalachia Educational Laboratory 
Post Office Box 1348 



Charleston, West Virginia 25325-1348 
304/347-0400 
800/624-9120 (toll-free) 
304/347-0487 (FAX) 
aelinfo@ael.org 
http: / / www.ael.org 



This publication is based on work sponsored wholly or in partby the Office of Educational Research 
and Improvement, U. S. Department of Education, under contract number RJ96006001. Its contents do 
not necessarily reflect the views of OERI, the Department, or any other agency of the U. S. Government. 
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INTRODUCTION 



The Leadership to Unify School Improvement Efforts (LUSIE) project, developed by 
QUEST staff at the Appalachia Educational Laboratory (AEL) and funded by the West Virginia 
Department of Education, recently completed 14 months of work with the schools in three West 
Virginia county districts. One team from each of 1 1 schools in Logan, Marshall, and Raleigh 
Counties worked together to create school plans for improvement. In addition, participants 
helped write a document to aid others in West Virginia involved in creating state-required 
Unified School Improvement Plans (USIPs). A total of 66 people participated in the LUSIE 
process. 

The school leadership teams, consisting of a cross-section of each school community, met 
at four retreats held in Charleston, West Virginia. The first two retreats, held in August and 
November 1996, provided participants with an AEL-designed framework for continuous 
improvement and many techniques for facilitating the development of school visions and goals. 
Teams then used the techniques in their communities in order to gather data about school and 
community members’ concerns and values. LUSIE participants wrote their draft school 
improvement plans at the third retreat in April 1997 with such data in mind. During the fourth 
retreat, held in July 1997, ten LUSIE participants wrote a supplemental guide for other West 
Virginia schools involved in school improvement efforts, titled Creating Energy for School 
Improvement. 

After this document was completed, project staff discussed with one of the two QUEST 
evaluators ways to gather participants’ impressions of the value of their experience with the 
LUSIE project. Staff were concerned to know how participants understood the process 
conceptually, as well as how much impact it had on their schools and lives. They were further 
interested in participant critiques of the LUSIE project. 

Consequently, the evaluator and staff decided to conduct a telephone interview several 
months after Creating Energy for School Improvement was completed. The evaluator would 
conduct the interviews as well as analyze and report the data. These data were intended to 
inform staff of the ways in which participants found the process useful and meaningful, as well 
as ways in which LUSIE might be improved. These data would further contribute to staff s 
ongoing learnings about the inquiry process and efforts toward continuous school improvement. 



In addition, this evaluation will serve as a component of the participative assessment that 
QUEST staff have envisioned. Such an assessment perspective aims to capture data quite 
different from that generally associated with more quantitative approaches. A participative 
assessment better apprehends interpersonal and group processes and can more fully solicit 
respondents’ thoughts, dilemmas, and perspectives. In addition, participative assessment is 
consistent with the framework for continuous improvement that QUEST staff embrace; self- 
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evaluation, inquiry, reflection, and ongoing assessment require the flexibility that qualitative 
methods allow. 

The audience for this evaluation is QUEST staff. First, this assessment will help staff 
determine in what ways they might improve the inquiry process. And second, it will enhance 
staff’s learnings and insights about continuous school improvement efforts. 
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METHODOLOGY 



Since face-to-face interviews were not viable in terms of time and resources, QUEST 
staff and the evaluator chose to conduct telephone interviews of LUSIE participants. This 
methodological staple for evaluators and social science researchers provides both the structure of 
a fixed schedule of questions and the flexibility to accomodate new topics the interviewee may 
introduce to the interaction (Becker & Geer, 1957; Denzin, 1989; Dillman, 1978; Miles & 
Huberman, 1994). 

The QUEST evaluator conducted telephone interviews with a sample of 23 LUSIE 
participants from all of the various role groups and counties involved. These 23 respondents 
comprised thirty- five percent of the total 66 LUSIE participants. The sample was purposive- 
interviewees were chosen based on their participation in at least two of the first three LUSIE 
retreats. Several participants also had attended the fourth meeting, at which the LUSIE 
document was written. The sample included one assistant superintendent, six principals, three 
students, one parent, and twelve teachers. 

Interviewees were contacted over the course of four days in September 1997. LUSIE 
participants were reached either at their places of employment or their homes. Parents were most 
difficult to reach; consequently, only one parent was interviewed. Each interview lasted 
approximately five to ten minutes. All of the LUSIE participants contacted were willing to be 
interviewed and to discuss their experiences. 

The interview schedule consisted of five questions. QUEST staff and the evaluator 
constructed the questions together, aiming to gather as much information as possible wiA 
minimum inconvenience to respondents. The first question was intended to solicit participants 
perceptions of what the LUSIE experience was. In other words, this question was designed to 
prompt respondents to discuss their notion of LUSIE conceptually. The second question sought 
to discover if LUSIE had facilitated any change within the participants’ schools, while the next 
question asked if participants’ involvement with LUSIE had rendered any personal change. 
Suggestions for improving the LUSIE project were solicited in the fourth question. Finally, the 
last question asked respondents to discuss the most important insight they gained fi-om their 
participation in LUSIE. 
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FINDINGS 



Participant Understanding of LUSIE 

The first interview question dealt with participants’ conceptions of what constituted the 
LUSIE project. Nine participants answered this question with a positive evaluation of their 
LUSIE experience, while one student assessed the LUSIE project as “kinda boring...teachers 
would like it.” The remaining 13 interviewees offered various conceptual accounts of LUSIE. 
These included notions of LUSIE as “a good opportunity to work with people firom other 
counties to bring in good ideas to make a plan for better schools”; “a way to help schools with 
problems get their programs better and help them achieve their goals”; “a collaborative effort 
among parents, teachers, students, and administrators to come up with ways to make schools 
better”; and “training to help anybody involved in education to (1) understand what USD? is and 
(2) learn methods to help people to communicate effectively to develop a plan.” 

Five participants reported that initially they were confused by LUSIE, unsure of its 
purpose and direction. But all five also said that they eventually felt quite pleased with the work 
they accomplished during their involvement with the project. Several participants additionally 
characterized LUSIE as a “learning experience.” Another put it this way: “It’s hard.” 



School and Personal Change 

The second and third interview questions were designed to ascertain participants’ 
perceptions of the impact LUSIE had in their schools and in their personal lives. Five 
respondents reported that LUSIE did not effect any personal change. Other particip^ts 
described a variety of personal changes: LUSIE “made me be more open-minded,” “gave me 
more insight into our needs,” **made me more aware of the importance of local schools, helped 
me be more involved with what’s happening to help my school be a better place, I now look'at 
education fi-om more than a teacher’s view,” “made me realize the value of working with other 
people,” “I could never see the whole picture - now I can.” One teacher also noted becoming 
more attuned to the school’s goals rather than only her goals for the classroom. 

As one interviewee put it, LUSIE ensured that school change happened intentionally, not 
accidentally.” Many participants noted that LUSIE gave their schools the tools to conduct needs 
assessments and to produce a school plan collaboratively. As a teacher reported, “I’m not sure 
our school would have gone that route [assessing needs and setting goals] without LUSIE.” One 
principal even asserted that without LUSIE’s assistance, her school *would have been at a loss 
when it came to writing an USIP. 

One participant noted that LUSIE “opened communications at school [because] the 
methods [used by LUSEE staff] unified us more.” Another reported that the “techniques brought 
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awareness of how [we] need to involve all [in school planning] rather than [relying on] a 
mandate from on high.” Yet another participant offered that LUSIE generated more teacher self- 
evaluation in the classroom. 

However, five respondents found that little or nothing had changed at their schools as a 
result of LUSIE. One respondent noted that “in the grand scheme, nothing’s changed.” 
Similarly, one teacher said, “I’d like to see us have a unified program, see us move toward these 
goals. But unfortunately it seems like a ball out of space no one knows what to do with.” And 
two students likewise felt that LUSIE had not contributed to much change in their schools. 



Participant Criticisms 

The fourth interview question asked participants to share their criticisms of the LUSIE 
project. Eight respondents said they had no criticisms. Most of the critiques the remaining 
interviewees offered had to do with the time and scheduling constraints inherent in any off- 
campus meeting. Some cited wanting more time to communicate with other participants, while 
others noted the problems associated with, as one respondents put it, trying to “jam so much 
information into a short time.” Another respondent mentioned that the process took such a long 
time that other teachers “thought I played for a year!” Finally, several particip^ts noted that 
LUSIE “had a slow start” and that “maybe [we] could have started writing earlier, in November 

rather than April.” 

Another criticism was that the meetings held at AEL’s offices were too crammed, that 

there was not enough room to move about comfortably. Several participants noted that they 
preferred the time spent at the John XXIH Pastoral Center, a retreat facility, for this reason. A 
final criticism mentioned by two participants was that LUSIE offered “too much theory, not 
enough practical application.” 



Important Insights 

The final interview question asked respondents to discuss the most important insight or 
learning experience they gained from their participation in LUSIE. Two participants answered 
that LUSIE had enabled them to become more organized and systematic, having learned how to 
collect data and set goals. As one student put it, “At my clubs, now I can say how we can reach 
our goals.” Belatedly, one principal expressed surprise and pleasure at his teachers’ ability to 
write the school plan. 

This principal also described having a new “appreciation for [the] techmcal awareness of 
people who aren’t in a school and their ability to understand some of the subtleties of our 
school.” Likewise, nine interviewees noted that they had learned from the various other 
participants with whom they might not have otherwise explored school improvement. 
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Other learning experiences were more personal. For instance, one principal said, “The 
Legislature says we have to do something [construct a plan]. It’s kinda intimidating. [The 
LUSIE facilitators] made it seem like something we do anyway, just a matter of doing it and 
getting it on paper.” This principal also very much enjoyed meeting state department employees 
informally within the LUSIE context: “They took a lot of the fear away. [I thought] ‘Oh, they’re 
just like me.’” 

The assistant superintendent noted that, through LUSIE, sequential (K-12) community 
involvement in the local schools developed where before there was none. One teacher responded 
that LUSIE had changed his perspective: LUSIE “makes you look inward as a teacher,” while 
another teacher offered that the most important learning experience gained from LUSIE was the 
opportunity to analyze her school and examine areas for improvement more objectively. Finally, 
one teacher offered that his school was confident that they had produced a worthy USIP because 
of their participation in LUSIE, while other schools not involved with the project were 
“scurrying around” to create their own plans. 
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CONCLUSIONS AND RECOMMENDATIONS 



Conclusions 

Most participants appeared to have a basic understanding of the LUSIE project. They 
understood, for instance, that they were gaining skills and techniques that would help them create 
a school improvement plan in collaboration with their school’s publics. Many other respondents 
simply said that they enjoyed the experience of participating in LUSIE. 

Some interviewees saw no change within their schools as a result of participation in 
LUSIE. Others expressed concern about the implementation of their school plans. As one 
participant put it, “We would have a better system, if we followed the plan.” On the other hand, 
some participants cited increased focus on school improvement, better orgamzation within their 
schools, and greater inclusion of the wider school community. Others simply stated that their 
schools now have a plan as a result of LUSIE. Only two participants were able to cite specific 
changes that had thus far been instituted in their schools. 

In terms of personal changes resulting fi:om participation in LUSIE, some interviewees 
mentioned their increased self-awareness and self-evaluation, as well as their improved 
receptiveness to others’ perceptions. Several respondents felt that their involvement in LUSIE 
had generated no personal changes. 

Most participants had no serious critiques of LUSIE. They were pleased with the process 
and felt their work had been productive. As an aside, some noted that they had hoped for more 
time to be allotted for interactions with colleagues, but acknowledged that this was a problem 
commonly associated with many meetings. Others mentioned difficulties with scheduling they 
had encountered, but these respondents also felt that such problems were inevitable. 

Other concerns, however, were more specific. More space in which to move around 
would have been helpfiil to some participants. Another felt that the group could have begun 
writing plans earlier in the process. Additionally, a few respondents were concerned about how, 
and even if, their school improvement plans would be implemented. 

Finally, many participants reported that one of the most important learning experiences 
they had as a result of their involvement in LUSIE was an increased awareness of the larger 
school community. Several respondents also noted that they had discovered or developed new 
competence in communicating and organizing change. 

The principals and administrator were quite pleased with LUSIE. Teachers, too, enjoyed 
the process and found it useful. They often noted how much they appreciated the opportunity to 
communicate with other educators and people concerned about education, as well as the time to 
reflect on their work. One student found the experience boring, while the remaining two students 
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interviewed saw little or no change in their schools following LUSIE. The one parent 
interviewed found LUSIE to be finitful and productive. 

In sum, most participants seemed pleased with LUSIE and their resultant school plans, 
although some noted their initial skepticism. Further, most respondents enjoyed meeting other 
people concerned about education to discuss school improvement. 



Recommendations 

Based on the information gathered from the telephone interviews of a sample of 23 
LUSIE participants, the following recommendations are made. 

First, students could be integrated more fully into LUSIE activities. The students 
interviewed all indicated that on some level they found the proceedings irrelevant or ineffective. 

Second, educators described concern about the actual unplementation of their school 
improvement plans. Perhaps follow-up activities could be undertaken to support schools as they 
attempt to institute some of the changes suggested in their plans. 

Third, facilitators perhaps could address the concerns of those who noted a disjuncture 
between theoretical considerations and the practical applications of the work accomplished as a 
result of the LUSIE project. 
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Interview Protocol 



Hello. This is Caitlin Howley-Rowe from Appalachia Educational Laboratory. May I please 
speak to ? 

Hello. I’m Caitlin Howley-Rowe from Appalachia Educational Laboratory. I m the third-party 
evaluator for the LUSIE project that you participated in, and I’m calling to ask you a few 
questions about your experiences with the project. You may have read in a recent letter from 
Beth Sattes that I would be calling you. 

I want to assure you that your name will not be associated with any of your comments, and 
anything you tell me will be used for our own learning as we try to improve this project. Also, 
this interview ought to take only 10 minutes or so. 

Is now a good time for you talk with me? 

(If yes, continue) 

Date; 

Time: 

Role group: 

1. How would you describe the LUSIE project to someone who had never encountered it before? 



2. In what ways, if any, did involvement with LUSIE change the way things are done at your 
school? 



3. In what ways, if any, did your experience with LUSIE change you personally? 



4. What criticisms, if any, of the LUSIE project do you have? 



5. What was the most important insight or learning experience you gained from your 
involvement in LUSIE? 
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